Systems and methods for segmentation of intra-patient medical images

ABSTRACT

Embodiments disclose a method and system for segmenting medical images. In certain embodiments, the system comprises a database configured to store a plurality of medical images acquired by an image acquisition device. The plurality of images include at least one first medical image of an object, and a second medical image of the object, each first medical image associated with a first structure label map. The system further comprises a processor that is configured to register the at least one first medical image to the second medical image, determine a classifier model using the registered first medical image and the corresponding first structure label map, and determine a second structure label map associated with the second medical image using the classifier model.

CROSS REFERENCE TO RELATED APPLICATION

This application claims the benefit of priority to U.S. ProvisionalApplication No. 62/281,652, filed on Jan. 21, 2016, the entire contentof which has been incorporated herein by reference.

TECHNICAL FIELD

This disclosure relates generally to image segmentation. Morespecifically, this disclosure relates to systems and methods foraccurate medical image segmentation using a patient's prior imageinformation to aid in the segmentation of the patient's succeedingimages.

BACKGROUND

Image segmentation techniques are widely used for segmenting medicalimages and determining contours between anatomical structures within theimages. For example, in radiation therapy, automatic segmentation oforgans is usually performed to reduce contouring time, and improvecontour accuracy and consistency over various hospitals. However,automated segmentation of images remains to be a very difficult task dueto noises, limited image contrast and/or low image quality. For example,medical images having lower image quality, such as some computertomography (CT) or cone-beam computer tomography (CBCT) images that maybe used to treat cancer patients, are known to have lower contrast andlittle texture for most soft tissue structures. Therefore, conventionalimage segmentation methods based primarily on image contrast often failto find an accurate contour between the background and anatomicalstructures (e.g., organs or tumors) of interest, or between differentanatomical structures in a medical image.

Two main methods of medical image segmentation include Atlas-basedauto-segmentation and statistical learning segmentation. Atlas-basedauto-segmentation (ABAS) registers an anatomy labeled image onto anunlabeled image and transfers the labels, wherein the labels identifythe anatomic structures in the images (e.g., prostate, bladder, rectumand the like). Statistical learning segmentation classifies image voxelsaccording to the voxel properties of the anatomic structures in theimages, wherein the voxel properties include, for example, intensity,texture features and the like.

FIG. 1 illustrates an exemplary three-dimensional (3D) CT image from atypical prostate cancer patient. Illustration (A) shows a pelvic regionof the patient in a 3D view, which includes the patient's bladder,prostate, and rectum. Images (B), (C), and (D) are axial, sagittal andcoronal views from a 3D CT image of this pelvic region. As shown inimages (B), (C), and (D), most part of the patient's prostate boundaryis not visible. That is, one cannot readily distinguish the prostatefrom other anatomical structures or determine a contour for theprostate. In comparison, images (E), (F), and (G) show the expectedprostate contour on the same 3D CT image. As illustrated in FIG. 1,conventional image segmentation methods based solely on contrast andtextures presented in the image would likely fail when used to segmentthis exemplary 3D CT image.

Recent developments in machine learning techniques make improved imagesegmentation on lower quality images possible. For example, supervisedlearning algorithms can “train” computers to predict the anatomicalstructure each pixel or voxel of a medical image represents. Suchprediction usually uses features of the pixel or voxel as inputs.Therefore, the performance of the segmentation highly depends on thetype of features available. To date, most learning-based imagesegmentation methods are based primarily on local image features such asimage intensities and image textures. As a result, these segmentationmethods are still suboptimal for lower quality images, such as the 3D CTimage shown in FIG. 1.

Accurate segmentation of anatomical structures from CT images remains achallenging problem. Segmented serial images of the same patient have aspecial utility in adaptive plan review/re-planning and doseaccumulation. Serial images necessarily sample different probabilitydistributions than those of populations and therefore ought to provideinformation that aids the segmentation of new serial images.

Some methods propose combining atlas and statistical methods to gainimproved segmentation accuracy. For example, one method bases itson-line learning and patient-specific classification onlocation-adaptive image contexts. The location-adaptive classifiers aretrained on static image appearance features and image context featuresas well. Such a method has been used on serial prostate images with thestated goal of refining the patient's prostate segmentation asradiotherapy progressed. However, it requires that the data be treatedas a three-dimensional data object using the spatial distribution of thevoxel patch-pairs as features themselves, and in addition, using arandom forest (RF) method.

Alternatively, another method proposes combining bladder volume metricand landmark locations with conventional deformable registration in astudy of atlas based segmentation of serial CT images of thecervix-uterus. However, the method can be computationally expensive anderror prone since use of landmarks requires human intervention.

Furthermore, another method uses random forests for brain tissuesegmentation (in a semi-supervised learning mode) such that the decisionforest is trained on anatomy-labeled pre-surgical imagery and unlabeledpost-surgical images of the same patient. However, this approach avoidsthe registration step and thus, fails to take advantage of usefulinformation in Atlas-based registration.

In addition, random forest segmentation has been combined with imageregistration to perform image segmentation. This allows forinter-patient segmentation, and through the mechanism of an “atlasforest”, multiple patients' images were registered to a commoncoordinate frame, with one image per forest model. However, the methodrequires different compositions of training data to provide resultingmodels.

Further, methods combining deformable registration and random forestsare used to segment the bones of the face and teeth from dental ConeBeam CT images. Patient images are registered to an atlas for an initialprediction or estimate of the segmentation. This is followed by asequence of random forest classifiers that use context features as partof the features to be trained on at each stage of the sequence. However,to the method requires using deformable registration and RF classifiersto form RF models based on prior images of the same patient.

The disclosed methods and systems are designed to solve one or moreproblems discussed above, by combining Atlas-based segmentation andstatistical learning, combining population with intra-patient image andstructure information, and enabling convenient updating of a patientsegmentation model in order to improve segmentation performance onmedical images in radiation therapy or related fields.

SUMMARY

Certain embodiments of the present disclosure relate to a system forsegmenting medical images. The system may include a database configuredto store a plurality of medical images acquired by an image acquisitiondevice. The plurality of medical images may include at least one firstmedical image of an object, and a second medical image of the object,each first medical image associated with a first structure label map.The system may further include a processor that is configured toregister the at least one first medical image to the second medicalimage, determine a classifier model using the registered first medicalimage and the corresponding first structure label map, and determine asecond structure label map associated with the second medical imageusing the classifier model.

Certain embodiments of the present disclosure relate to acomputer-implemented method for segmenting medical images. The methodmay include the operations performed by at least one processor. Theoperations may include receiving at least one first medical image of anobject, and a second medical image of the object, from a databaseconfigured to store a plurality of medical images acquired by an imageacquisition device, each first medical image associated with a firststructure label map. The operations may further include registering theat least one first medical image to the second medical image,determining a classifier model using the registered first medical imageand the corresponding first structure label map, and determining asecond structure label map associated with the second medical imageusing the classifier model.

Certain embodiments of the present disclosure relate to a non-transitorycomputer-readable medium containing instructions that, when executableby at least one processor, cause the at least one processor to perform amethod for segmenting medical images. The method may include receivingat least one first medical image of an object, and a second medicalimage of the object, from a database configured to store a plurality ofmedical images acquired by an image acquisition device, each firstmedical image associated with a first structure label map. The methodmay further include registering at least one first medical image to thesecond medical image, determining a classifier model using theregistered first medical image and the corresponding first structurelabel map, and determining a second structure label map associated withthe second medical image using the classifier model.

Additional features and advantages of the present disclosure will be setforth in part in the following description, and in part will be obviousfrom the description, or may be learned by practice of the presentdisclosure. The features and advantages of the present disclosure willbe realized and attained by means of the elements and combinationspointed out in the appended claims.

It is to be understood that the foregoing general description and thefollowing detailed description are exemplary and explanatory only, andare not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which constitute a part of thisspecification, illustrate several embodiments and, together with thedescription, serve to explain the disclosed principles.

FIG. 1 illustrates an exemplary 3D CT image from a typical prostatecancer patient.

FIG. 2A is a block diagram showing an exemplary radiotherapy system 100,according to some embodiments of the present disclosure.

FIG. 2B depicts an exemplary image-guided radiotherapy device, accordingto some embodiments of the present disclosure.

FIG. 3 illustrates an exemplary image segmentation system, according tosome embodiments of the present disclosure.

FIG. 4 illustrates an exemplary medical image processing device,according to some embodiments of the present disclosure.

FIG. 5 is a flowchart illustrating an exemplary image segmentationprocess, according to some embodiments of the present disclosure.

FIG. 6 is a flowchart illustrating an exemplary training method,according to some embodiments of the present disclosure.

DETAILED DESCRIPTION

While examples and features of disclosed principles are describedherein, modifications, adaptations, and other implementations arepossible without departing from the spirit and scope of the disclosedembodiments. Also, the words “comprising,” “having,” “containing,” and“including,” and other similar forms are intended to be equivalent inmeaning and be interpreted as open ended, such that an item or itemsfollowing any one of these words is not meant to be an exhaustivelisting of the item or items, or meant to be limited to only the listeditem or items. And the singular forms “a,” “an,” and “the” are intendedto include plural references, unless the context clearly dictatesotherwise.

Systems and methods consistent with the present disclosure are directedto segmenting a medical image of an object using learning algorithmstrained using mapped atlases derived based on prior images of the sameobject. Specifically, embodiments of the present disclosure providevarious combinations of atlas-based auto-segmentation (ABAS) and RandomForest (RF) segmentations of serial CT images, incorporatingpatient-specific RF models learned on prior-day images and theirstructures.

As used herein, a “learning algorithm” refers to any algorithm that canlearn a model or a pattern based on existing information or knowledge.The learned model or pattern can be then used to predict or estimateoutput using input of new information or knowledge. For example, thelearning algorithm may be a machine learning algorithm or any othersuitable learning algorithm. In some embodiments, a supervised learningalgorithm, such as a Support Vector Machine (SVM), Adaboost/Logitboost,Random Forests, and neural network (e.g., a convolutional neuralnetwork), may be used. In some other embodiments, semi-supervisedlearning algorithms may also be used.

Supervised learning is a branch of machine learning that infers aprediction model given a set of training data. Each individual sample ofthe training data is a pair containing a data vector (such as a seriesof measurements) and a desired output value. A supervised learningalgorithm analyzes the training data and produces a predictor function.The predictor function is called a classifier or a classification modelwhen the output is discrete, such as a list of labels identifyingdifferent groups. The predictor function, once derived through training,is capable of predicting the correct output value for a valid input.

Consistent with the disclosed embodiments, image segmentation may beformulated as a learning-based classification function, which classifieseach image point of the medical image into one of the anatomicalstructures. As used herein, an “image point” refers to an image elementin a digital image that corresponds to a physical point in theunderlying object. For example, the image point is a pixel in a 2D imageor a voxel in a 3D image.

Consistent with the disclosed embodiments, the image segmentation mayalso classify image blocks rather than image points. As used herein, an“image block” is a group of image points to be classified together. Forexample, the image block may be a super-pixel in a 2D image, or asuper-voxel in a 3D image. When image points within an image block areknown to belong to the same anatomical structure, classifying based onimage blocks may be more efficient and accurate. Accordingly, wheneverthe term “image point” is used throughout this disclosure, it intends tocover both the basic “image point” and also the “image block” as definedabove.

The disclosed systems and methods provide an estimated structure labelmap for the image. A label map refers to a map of structure labels eachidentifying a corresponding image point as being within a particularstructure of interest. Alternatively, consistent with this disclosure, alabel map may also be a probability map, which contains structure labelsthat each represents the probability of the image point belonging to thestructure. For example, when segmenting an image including multiplestructures, a structure label of an image point may provide a set ofprobability values indicting how likely the image point belonging toeach of the structures under consideration.

Consistent with the disclosed embodiments, the classifier is trainedusing a set of training images. As used herein, a “training image” is animage where the image points are already classified and labeled. Forexample, a training image may be an atlas. As used consistently herein,an “atlas” includes an image and corresponding structure delineations(annotations) indicating what structure(s) the image points belong to.The image in an atlas, also referred to as an atlas image, can be aprevious image of the subject patient taken at an earlier time. Thestructure delineations can be represented as, for example, structurelabel maps, structure surfaces, or structure contours. The descriptionbelow uses the label maps as an example of the structure delineationsand is similarly applied to the scenarios of structure surfaces andcontours.

Consistent with the disclosed embodiments, the training process usesfeatures extracted from the atlas image and the corresponding structuredelineations to train a classifier. Once properly trained using theprocess discussed in more detail below, such an algorithm can be used tosegment a new image.

Consistent with the disclosed embodiments, the atlases used in thetraining process may be derived from prior images of the same object asthe new image to be segmented. Similarity between such training imagesand the new image to be segmented can improve the segmentation accuracyand efficiency. In some embodiments, the atlases used to train aclassifier may be obtained by segmenting a previous image using apreviously trained classifier. In some embodiments, the atlases used maybe registered to the current image.

Consistent with the disclosed embodiments, intra-patient images obtainedduring the course of radiation therapy are used. The intra-patientimages are typically obtained one or more days between images to checkon the patient's response to the prescribed therapy. Delineating theanatomy of serial images of the same patient's anatomy using thepatient's prior-image information aids in the segmentation of succeedingimages. Accurate serial image segmentation is a prerequisite foradaptive therapy plan assessment/re-planning and dose accumulation.

Consistent with the disclosed embodiments, the combination ofAtlas-based auto-segmentation (ABAS) and statistical learning (SL)segmentations of serial CT images may improve segmented serial images ofthe same patient during adaptive planning and determining doseaccumulation. This combination may also improve the prediction accuracyand thus the quality of image segmentation. In addition, the use ofprior days' images and structures deformably registered to succeedingdays' images to form statistical models may provide more accuratesegmentations than models based on unregistered images.

Atlas-based segmentation registers an image with anatomy labels attachedto its voxels (the atlas) to a target image, and then assigns the atlaslabels to the corresponding target voxels. Statistical learning assignstarget voxel labels using a classifier program in which labels areassociated with regions of a feature space. The features include voxelintensity, appearance (local variation measures), and structuralproperties of the images. The label-feature association is learned bytraining the classifier program on labeled images.

In an embodiment, the statistical learning method used is the RandomForests (RF) method. A random forest is a set of decision trees.Starting with a random sample of the training data (such as voxelaverage intensities for voxel patches, and pairs of patches locatedrandomly around the index voxel), and a random selection of variablesembedded in the data, the method determines the best parameter split ofthe samples into the label categories, splits the data sample, and thenfollows the split data down to the next pair of nodes where the bestsplit is found. In an embodiment, this method is recursively performeduntil terminating at the maximum tree depth, or until a minimum samplesize is reached. The terminal tree nodes (e.g., leaves) provide a labelassignment. Trees thus trained perform classification when new featuredata is processed through the trees' nodes. Multiple trees increase thediscriminative power of the classifier. Random forest classifiers resistoverfitting, and can be used for efficient image segmentations.Segmentation methods incorporating patient-specific RF models learned onprior-day images and their structures provide more accurate segmentationthan methods than methods that solely use a population model.

Although the combination of ABAS and RF segmentations is used as anexemplary combination consistent with the present disclosed, otherstatistical learning algorithms and various combinations with ABAS arecontemplated, including those learning algorithms discussed above.Further, the disclosed image segmentation systems and methods can beapplied to segment medical images obtained from any type of imagingmodalities, including, but not limited to X-ray, CT, CBCT, spiral CT,magnetic resonance imaging (MRI), ultrasound (US), positron emissiontomography (PET), single-photon emission computed tomography (SPECT),and optical images. Furthermore, the disclosed image segmentationsystems and methods can be adapted to segment both 2D and 3D images.

Exemplary embodiments are now described with reference to theaccompanying drawings. Wherever convenient, the same reference numbersare used throughout the drawings to refer to the same or like parts.

FIG. 2A is a block diagram showing an exemplary radiotherapy system 100,according to some embodiments of the present disclosure. Radiotherapysystem 100 may be an IGRT system. As shown in FIG. 2A, radiotherapysystem 100 may include a control console 110, a database 120, aradiotherapy device 130, and an image acquisition device 140. In someembodiments, radiotherapy device 130 and image acquisition device 140may be integrated into a single image-guided radiotherapy device 150, asindicated by the dashed box 150 in FIG. 2A. In some embodiments,radiotherapy device 130 and image acquisition device 140 may be separatedevices. In some embodiments, radiotherapy device 130 and imageacquisition device 140 may be physically or communicative connected toeach other, as indicated by a dotted-dashed line between radiotherapydevice 130 and image acquisition device 140 in FIG. 2A.

Control console 110 may include hardware and software components tocontrol radiotherapy device 130 and image acquisition device 140 and/orto perform functions or operations such as treatment planning, treatmentexecution, image acquisition, image processing, motion tracking, motionmanagement, or other tasks involved in a radiotherapy process. Thehardware components of control console 110 may include one or morecomputers (e.g., general purpose computers, workstations, servers,terminals, portable/mobile devices, etc.); processor devices (e.g.,central processing units (CPUs), graphics processing units (GPUs),microprocessors, digital signal processors (DSPs), field programmablegate arrays (FPGAs), special-purpose or specially-designed processors,etc.); memory/storage devices (e.g., read-only memories (ROMs), randomaccess memories (RAMs), flash memories, hard drives, optical disks,solid-state drives (SSDs), etc.); input devices (e.g., keyboards, mice,touch screens, mics, buttons, knobs, trackballs, levers, handles,joysticks, etc.); output devices (e.g., displays, printers, speakers,vibration devices, etc.); or other suitable hardware. The softwarecomponents of control console 110 may include operation system software,application software, etc. For example, as shown in FIG. 2A, controlconsole 110 may include treatment planning/delivery software 115 thatmay be stored in a memory/storage device of control console 110.Software 115 may include computer readable and executable codes orinstructions for performing the processes described in detail below. Forexample, a processor device of control console 110 may becommunicatively connected to a memory/storage device storing software115 to access and execute the codes or instructions. The execution ofthe codes or instructions may cause the processor device to performoperations to achieve one or more functions consistent with thedisclosed embodiments.

Control console 110 may be communicatively connected to database 120 toaccess data. In some embodiments, database 120 may be implemented usinglocal hardware devices, such as one or more hard drives, optical disks,and/or servers that are in the proximity of control console 110. In someembodiments, database 120 may be implemented in a data center or aserver located remotely with respect to control console 110. Controlconsole 110 may access data stored in database 120 through wired orwireless communication.

Database 120 may include patient data 122. Patient data may includeinformation such as (1) imaging data associated with a patientanatomical region, organ, or volume of interest segmentation data (e.g.,MRI, CT, X-ray, PET, SPECT, and the like); (2) functional organ modelingdata (e.g., serial versus parallel organs, and appropriate dose responsemodels); (3) radiation dosage data (e.g., may include dose-volumehistogram (DVH) information); or (4) other clinical information aboutthe patient or course of treatment.

Database 120 may include machine data 124. Machine data 124 may includeinformation associated with radiotherapy device 130, image acquisitiondevice 140, or other machines relevant to radiotherapy, such asradiation beam size, arc placement, on/off time duration, radiationtreatment plan data, multi-leaf collimator (MLC) configuration, MRIpulse sequence, and the like.

Image acquisition device 140 may provide medical images of a patient.For example, image acquisition device 140 may provide one or more of MRIimages (e.g., 2D MRI, 3D MRI, 2D streaming MRI, 4D volumetric MRI, 4Dcine MRI); Computed Tomography (CT) images; Cone-Beam CT images;Positron Emission Tomography (PET) images; functional MRI images (e.g.,fMRI, DCE-MRI, diffusion MRI); X-ray images; fluoroscopic images;ultrasound images; radiotherapy portal images; Single-Photo EmissionComputed Tomography (SPECT) images; and the like. Accordingly, imageacquisition device 140 may include an MRI imaging device, a CT imagingdevice, a PET imaging device, an ultrasound imaging device, afluoroscopic device, a SPECT imaging device, or other medical imagingdevices for obtaining the medical images of the patient.

Radiotherapy device 130 may include a Leksell Gamma Knife, a linearaccelerator or LINAC, or other suitable devices capable of deliveringradiation to an anatomical region of interest of a patient in acontrollable manner.

FIG. 2B depicts an exemplary image-guided radiotherapy system 200,consistent with disclosed embodiments. Consistent with some embodiments,the disclosed image segmentation system may be part of a radiotherapysystem as described with reference to FIG. 2A. As shown, system 200 mayinclude a couch 210, an image acquisition device 220, and a radiationdelivery device 230. System 200 delivers radiation therapy to a patientin accordance with a radiotherapy treatment plan.

Couch 210 may support a patient (not shown) during a treatment session.In some implementations, couch 210 may move along a horizontal,translation axis (labelled “I”), such that couch 210 can move thepatient resting on couch 210 into and/or out of system 200. Couch 210may also rotate around a central vertical axis of rotation, transverseto the translation axis. To allow such movement or rotation, couch 210may have motors (not shown) enabling the couch to move in variousdirections and to rotate along various axes. A controller (not shown)may control these movements or rotations in order to properly positionthe patient according to a treatment plan.

In some embodiments, image acquisition device 220 may include an MRImachine used to acquire 2D or 3D MRI images of the patient before,during, and/or after a treatment session. Image acquisition device 220may include a magnet 221 for generating a primary magnetic field formagnetic resonance imaging. The magnetic field lines generated byoperation of magnet 221 may run substantially parallel to the centraltranslation axis I. Magnet 221 may include one or more coils with anaxis that runs parallel to the translation axis I. In some embodiments,the one or more coils in magnet 221 may be spaced such that a centralwindow 223 of magnet 221 is free of coils. In other embodiments, thecoils in magnet 221 may be thin enough or of a reduced density such thatthey are substantially transparent to radiation of the wavelengthgenerated by radiotherapy device 230. Image acquisition device 220 mayalso include one or more shielding coils, which may generate a magneticfield outside magnet 221 of approximately equal magnitude and oppositepolarity in order to cancel or reduce any magnetic field outside ofmagnet 221. As described below, radiation source 231 of radiotherapydevice 230 may be positioned in the region where the magnetic field iscancelled, at least to a first order, or reduced.

Image acquisition device 220 may also include two gradient coils 225 and226, which may generate a gradient magnetic field that is superposed onthe primary magnetic field. Coils 225 and 226 may generate a gradient inthe resultant magnetic field that allows spatial encoding of the protonsso that their position can be determined. Gradient coils 225 and 226 maybe positioned around a common central axis with the magnet 221, and maybe displaced along that central axis. The displacement may create a gap,or window, between coils 225 and 226. In the embodiments wherein magnet221 also includes a central window 223 between coils, the two windowsmay be aligned with each other. In some embodiments, image acquisitiondevice 220 may be an imaging device other than an MRI, such as an X-ray,a CT, a CBCT, a spiral CT, a PET, a SPECT, an optical tomography, afluorescence imaging, ultrasound imaging, or radiotherapy portal imagingdevice.

Radiotherapy device 230 may include the source of radiation 231, such asan X-ray source or a linear accelerator, and a multi-leaf collimator(MLC) 233. Radiotherapy device 230 may be mounted on a chassis 235. Oneor more chassis motors (not shown) may rotate chassis 235 around couch210 when couch 210 is inserted into the treatment area. In anembodiment, chassis 235 may be continuously rotatable around couch 210,when couch 210 is inserted into the treatment area. Chassis 235 may alsohave an attached radiation detector (not shown), preferably locatedopposite to radiation source 231 and with the rotational axis of chassis335 positioned between radiation source 231 and the detector. Further,device 230 may include control circuitry (not shown) used to control,for example, one or more of couch 210, image acquisition device 220, andradiotherapy device 230. The control circuitry of radiotherapy device230 may be integrated within system 200 or remote from it.

During a radiotherapy treatment session, a patient may be positioned oncouch 210. System 200 may then move couch 310 into the treatment areadefined by magnetic coils 221, 225, 226, and chassis 235. Controlconsole 240 may then control radiation source 231, MLC 233, and thechassis motor(s) to deliver radiation to the patient through the windowbetween coils 225 and 226 according to a radiotherapy treatment plan.

FIG. 3 illustrates an exemplary image segmentation system 300 forsegmenting medical images, according to some embodiments of the presentdisclosure. Image segmentation system 300 may include a medical imagedatabase 301, a classifier training unit 302, a structure classificationunit 303, and a network 305. In some embodiments, image segmentationsystem 300 may include more or less of the components shown in FIG. 3.For example, when an anatomical classifier is pre-trained and provided,image segmentation system 300 may only include structure classificationunit 303, medical image database 301, and, optionally, network 305.

In some embodiments, the various components of image segmentation system300 may locate remotely from each other, and be connected throughnetwork 305. In some alternative embodiments, certain components ofimage segmentation system 300 may be located on the same site or insideone device. For example, medical image database 301 may be located onsite with classifier training unit 302, or be part of classifiertraining unit 302. As another example, classifier training unit 302 andstructure classification unit 303 may be inside the same computer orprocessing device.

Image segmentation system 300 may be used to segment serialintra-patient CT images stored in medical image database 301. Consistentwith the disclosure, “serial images” may be images acquired duringserial radiotherapy treatment sessions of a patient. The serialradiotherapy treatment sessions may be performed at a set frequency(e.g., every day, every week, etc.) or at discrete time points,according to a radiotherapy treatment plan for the patient. The serialimages may include current images and prior images. In some embodiments,a “current image” may be a current day medical image of a patient, e.g.,an image taken during the treatment session of a patient that occurredon the present day. As shown in FIG. 3, classifier training unit 302 maycommunicate with medical image database 301 to receive one or more“prior images” of the same patient. A “prior image” may be an imagetaken during a treatment session of the same patient but occurred on aprevious day. The prior images stored in medical image database 301 maybe obtained from a medical image database, which contain images ofprevious radiotherapy treatment sessions.

In some embodiments, the prior images may be pre-segmented. For example,the prior images may be segmented automatically by image segmentationsystem 300, or manually by user 313. User 313 may be an expert, e.g., aradiologist or another physician familiar with anatomical structures inmedical images, who provides expert structure label maps associated withthe prior images. In that case, the prior images and their correspondingstructure label maps become atlases that can be readily used byclassifier training unit 302 to train a structure classifier.

If the prior images are not pre-segmented, they may be sent to structureclassification unit 303 for segmentation. In some embodiments, structureclassification unit 303 may use the most recently trained classifier tosegment the prior images, or a merged classifier of selected previouslytrained classifiers for the segmentation. Structure classification unit303 may provide a structure label map for each prior image. Structureclassification unit 303 may then provide the atlases, consisting of theprior images along with their corresponding structure label maps, toclassifier training unit 302, as training images.

Classifier training unit 302 may use the training images received frommedical image database 301 to generate a structure classifier usinglearning algorithms. As shown in FIG. 3, classifier training unit 302may include an atlas registration module 321, a feature extractionmodule 322 and a training module 323. Classifier training unit 302 mayadditionally include input and output interfaces (not shown) tocommunicate with medical image database 301, network 305, and/or user312. Consistent with some embodiments, classifier training unit 302 maybe implemented with hardware (e.g., as disclosed in FIG. 4) speciallyprogrammed by software that performs an anatomical classifier trainingprocess.

Atlas registration module 321 may register the prior images to thecurrent images. Image registration is a process that transformsdifferent sets of data into one coordinate system. Typical imageregistration algorithms are either intensity-based or feature-based, orthe combination of the two. In particular, feature-based methods findcorrespondence between image features such as points, lines, andcontours. In some embodiments, the registration process may includemapping the image points of an atlas image to the image points of acurrent image. In some alternative embodiments, the registration processmay include mapping both the atlas images and the current image to areference image. In these embodiments, the reference image can be, forexample, an average atlas image or a common template image. As such, theatlas images are “indirectly” mapped to the current image. Various imageregistration methods can be used, such as one or a combination of any ofa linear registration, an object-driven “poly-smooth” non-linearregistration, or a shape-constrained dense deformable registration. Byperforming the image registration, an image transformation from theatlas image to the reference image is calculated for each atlas.

Atlas registration module 321 may further map the delineations (e.g.,structure labels) of each atlas to the space of the reference imageusing the corresponding image transformation for the atlas. The mappedstructure labels represent independent classification data, i.e.,independent segmentation results, of the current image from thecorresponding atlas.

A mapped atlas image and corresponding mapped structure labelsconstitute a mapped atlas, also referred to as a “registered atlas.” Insome embodiments, classifier training unit 302 may use the mappedatlases to train the structure classifier for segmenting the currentimages. Alternatively, classifier training unit 302 may use the mappedatlas images along with expert structure label maps for the training.

Feature extraction module 322 may determine and derive, for eachselected image point, one or more features such as image intensity,image texture, image patch, and curvature of an intensity profile. Thisfeature extraction process may repeat for a set of selected image pointsin a training image until all image points in the training image havebeen selected and processed.

Training module 323 may use the selected image points as training data,to train the classifier. In some embodiments, the training may be basedon learning algorithms, such as supervised machine learning algorithms.For example, learning algorithms such as Support Vector Machine (SVM),Adaboost/Logitboost, Random Forests, and Neural Networks may be used.The classifier is trained such that when the features for a particularimage point in the training image are input to the model, the modeloutputs a prediction of the anatomical structure that matches thepredetermined structure label of the image point. After being trainedusing numerous image points from numerous training images, theclassifier becomes competent enough to predict the anatomical structureof an unclassified image point in any new image.

Structure classification unit 303 may receive the trained structureclassifier from classifier training unit 302. As shown in FIG. 3,structure classification unit 303 may include a feature extractionmodule 331 and a classification module 332. Structure classificationunit 303 may additionally include input and output interfaces (notshown) to communicate with medical image database 301, network 305 anduser 313. Consistent with some embodiments, structure classificationunit 303 may be implemented with hardware (e.g., as disclosed in FIG. 4)specially programmed by software that performs an anatomical classifiertraining process.

Structure classification unit 303 may communicate with medical imagedatabase 301 to receive one or more current images. The current imagesmay be of a same object as the prior images. Feature extraction module331 may have similar hardware and software structures as featureextraction module 322. Feature extraction module 331 may identify one ormore features on each current image received from medical image database301. The features extracted by feature extraction module 331 may be thesame or similar to those used during the training stage by featureextraction module 322. The determined features may be provided toclassification module 332.

Classification module 332 may use the trained structure classifierreceived from classifier training unit 302, and the features receivedfrom feature extraction module 331, to predict the structure labels forthe respective image points. When all the selected image points areclassified, classification module 332 may output the segmented image. Insome embodiments, the segmented image may be displayed to user 313,and/or provided to treatment planning/delivery software 115 for furthertreatment usage. In some embodiments, the segmented image may beautomatically stored in medical image database 301 and become a priorimage.

Network 305 may provide connections between any of the above describedcomponents in image segmentation system 300. For example, network 305may be a local area network (LAN), a wireless network, a cloud computingenvironment (e.g., software as a service, platform as a service,infrastructure as a service), a client-server, a wide area network(WAN), and the like.

FIG. 4 illustrates an exemplary medical image processing device 400,according to some embodiments of the present disclosure. Medical imageprocessing device 400 may be an embodiment of classifier training unit302, structure classification unit 303, or their combination. In anotherembodiment, processing device 400 may be integrated into control console110 or radiotherapy device 230, shown in FIGS. 2A-(B). In someembodiments, medical image processing device 400 may be aspecial-purpose computer, or a general-purpose computer. For example,medical image processing device 400 may be a computer custom built forhospitals to handle image acquisition and image processing tasks.

As shown in FIG. 4, medical image processing device 400 may include aprocessor 421, a memory 422, a database 425, a data storage device 426,an input/output interface 427, a network interface 428, and a display429. Components of processing device 400 may be connected via a BUS.

Processor 421 may be one or more general-purpose processing devices,such as a microprocessor, central processing unit (CPU), graphicsprocessing unit (GPU), or the like. More particularly, processor 421 maybe a complex instruction set computing (CISC) microprocessor, reducedinstruction set computing (RISC) microprocessor, very long instructionWord (VLIW) microprocessor, a processor implementing other instructionsets, or processors implementing a combination of instruction sets.Processor 421 may also be one or more special-purpose processing devicessuch as an application specific integrated circuit (ASIC), a fieldprogrammable gate array (FPGA), a digital signal processor (DSP), aSystem on a Chip (SoC), or the like. Processor 421 may becommunicatively coupled to memory 422 and configured to execute thecomputer executable instructions stored thereon.

Memory 422 may include a read-only memory (ROM), a flash memory, arandom access memory (RAM), and a static memory, for example. In someembodiments, memory 422 may store computer executable instructions, suchas one or more image processing programs 423, as well as data used orgenerated while executing the computer programs, such as medical imagedata 424. Processor 421 may execute image processing programs 423 toimplement functionalities of anatomical classifier training unit 302and/or structure classification unit 303. Processor 421 may alsosend/receive medical image data 424 from memory 422. For example,processor 421 may receive prior image data or current image data storedin memory 422. Processor 421 may also generate intermediate data such asimage features and structure labels, and send them to memory 422.

Medical image processing device 400 may optionally include a database425, which may include or in communication with medical image database301. Database 425 may include a plurality of devices located either in acentral or distributed manner. Processor 421 may communicate withdatabase 425 to read images into memory 422 or store segmented imagesfrom memory 422 to medical image data 424.

Data storage device 426 may be an additional storage available to storedata associated with image processing tasks performed by processor 421.In some embodiments, data storage device 426 may include amachine-readable storage medium. While the machine-readable storagemedium in an embodiment may be a single medium, the term“machine-readable storage medium” should be taken to include a singlemedium or multiple media (e.g., a centralized or distributed database,and/or associated caches and servers) that store the one or more sets ofcomputer executable instructions or data. The term “machine-readablestorage medium” shall also be taken to include any medium that iscapable of storing or encoding a set of instructions for execution bythe machine and that cause the machine to perform any one or more of themethodologies of the present disclosure. The term “machine-readablestorage medium” shall accordingly be taken to include, but not belimited to, solid-state memories, optical and magnetic media.

Input/output 427 may be configured to allow data to be received and/ortransmitted by medical image processing device 400. Input/output 427 mayinclude one or more digital and/or analog communication devices thatallow processing device 400 to communicate with user or other machinesand devices. For example, input/output 427 may include a keyboard and amouse for user 312 or user 313 to provide input.

Network interface 428 may include a network adaptor, a cable connector,a serial connector, a USB connector, a parallel connector, a high-speeddata transmission adaptor such as fiber, USB 3.0, thunderbolt, and thelike, a wireless network adaptor such as a WiFi adaptor, atelecommunication (3G, 4G/LTE and the like) adaptor, and the like.Medical image processing device 400 may be connected to network 305through network interface 428. Display 429 may be any display devicethat suitable for displaying the medical images. For example, display429 may be an LCD, CRT, or LED display.

In some embodiments, the segmented current image may be displayed to auser on display 428. In some embodiments, the segmented current imagemay be provided to treatment planning/delivery software 115 for futuremedical usage and/or stored in medical image database 301 for futureimage segmentation tasks.

An accurately segmented image, or a well-defined contour of ananatomical structure may benefit various applications that rely onsegmentation results. For example, structure classification results mayalso help generate accurate estimation of volume size of the anatomicalstructure. For certain anatomical structures, such as the bladder,volume sizes are important in calculating the deformation field and doseoptimization for treatments. In the example of bladder, the volume sizesmay change significantly at different treatment sessions. Therefore,accurate estimation of its size will give important prior knowledgeabout the relative locations or deformations around the bladder, andthus help calculate the deformation field or optimize dose distributionon the fly.

FIG. 5 is a flowchart illustrating an exemplary image segmentationmethod 500 using combined atlas-based segmentation and statisticallearning segmentation. In some embodiments, method 500 may be performedby components of image segmentation system 300, such as classifiertraining unit 302 and structure classification unit 303. It iscontemplated that method 500 can be applied to segment one structure ofinterest or a group of structures of interest at the same time, such asa bladder, prostate, and rectum, which are spatially adjacent and highlycorrelated. Various machine learning methods, such as RF method, canhandle segmentation of multiple structures at the same time. Amulti-structure classifier model may be beneficial when the multiplestructures are spatially adjacent and thus highly correlated.

As shown in FIG. 5, at 501, medical image processing device 400 mayreceive an initial day medical image I₁ from a database, such as medicalimage database 301. In some embodiments, medical image I₁ represents aprior day medical image of a patient at day one.

At 502, medical image processing device 400 may identify at least onefeature from the day one image I₁. The at least one feature may beidentified by feature extraction module 332 of structure classificationunit 303, for example. The features can be the same types of featuresused for training the structure classifier, as will be described in moredetail below in connection with FIG. 6. Various methods may be used tocompute the attributes, including using machine learning models such asconvolutional neural network models.

At 503, the day one image I₁ is segmented by a population-trainedclassifier model M₁. For example, the day one image may be segmented bya random forests model M₁ trained on training images from a populationof patients. The day one image I₁ may be segmented by thepopulation-trained M₁ to produce a structure label map S₁, e.g.,representing the prostate, bladder, rectum, and/or other organs.

At 504, medical image processing device 400 may receive the j-thsucceeding day medical image I_(j) from a database, such as medicalimage database 301. Medical image I_(j) represents a succeeding daymedical image of the same patient at day two, or any succeeding j-th daywhere j=2, . . . . In some embodiments, day one image I₁ and succeedingday medical image I_(j) are acquired during serial radiotherapytreatment sessions of the same patient. For example, the medical imagesmay be acquired by image acquisition device 220 during successivetreatment therapy sessions delivered by image-guided radiotherapy system200.

At 505, the prior day image I_(j-1), j=2, . . . and its correspondingstructure label map S_(j-1) from step 603 may be used as an atlas andregistered to the current day image I_(j) to obtain a registered imageI_(j) ^((j-1)) and a corresponding atlas-based segmentation structurelabel map S_(j) ^((j-1)). For example, I₁, S₁ is used as an atlas todeformably register I₁ to day two image I₂ to obtain the registeredimage I₂ ⁽¹⁾ and day two structure label map S₂ ⁽¹⁾. The superscriptrefers to the atlas-day number, and serves to distinguish thepreliminary segmentation result S_(j) ^((j-1)) from the full-modelclassification S_(j) described below in 508.

At 506, a classifier model M_(temp) is trained on the image I_(j)^((j-1)) and the structure map S_(j) ^((j-1)) obtained as describedabove in step 505. In some embodiments, the model may be trainedaccording to an exemplary training method described in FIG. 6. Forexample, a new random forest model M_(temp) may be trained on the setI_(j) ^((j-1)), S_(j) ^((j-1)).

At 507, the trained new classifier model M_(temp) is merged with theprior day classifier model M_(j-1) to obtain a new classifier modelM_(j). Various methods may be used to merge the models. For example, themodel parameters may be determined based on the model parameters ofmodel M_(temp) and model M_(j).

At 508, the combined model is used to segment the current day imageI_(j), to produce the structure label map S_(j). This Structure labelmap S_(j) is derived from the model M_(j) while the structure label mapS_(j) ^((j-1)) is obtained by atlas-based segmentation using thepreceding day data I_(j-1), S_(j-1). As part of 508, feature extractionsmay be performed on the j-th prior day image, similar to 502.

At 509, the method determines whether all succeeding prior day imagesfor that patient have been used for training the classifier model. Ifthere are more succeeding prior day images for that patient, medicalimage processing device 400 repeats steps 504-509 to train model M_(j)(where j represents the j-th treatment day) until all the succeedingprior day images are processed. At 510, the segmentation of currentimage day I_(j) is complete.

The model for any j-th treatment day, M_(j), is obtained by training theclassifier on data that is the union of the registered prior day imagesand their corresponding structure label maps I_(i) ^((i-1)), S_(j)^((j-1)), denoted by

$\left\{ {\bigcup\limits_{i = 2}^{j}\left( {I_{i}^{({i - 1})},S_{i}^{({i - 1})}} \right)} \right\},{j \geq 2},{j = {{current}\mspace{14mu} {{day}.}}}$

The training is done in stages, one prior day per stage, resulting in amodel M_(temp) that is then combined with prior day's models, as thestatistical learning method permits.

In other embodiments the training data ensemble could contain otherforms of the structure label maps than that derived from the atlas-basedsegmentation. For example, the structure label map could be atlas-basedsegmentation structures corrected by expert programs or by humanexperts.

In other embodiments, the population based model M₁ (here included inthe resulting model M_(j)) could be dissociated from the models M₂ toM_(j).

In other embodiments, the composition of the training data could bevaried such that the training data for each j-th treatment day isaccumulated from the beginning of treatment and a model created from allthe accumulated training data, rather than sequential merging successivedays' trained models. This accommodates statistical learning methodssuch as neural networks that, unlike random or decision forests, havemodels that are not easily decomposed into daily data-trainedincrements.

In some embodiments, method 500 may be performed prior to radiotherapytreatment delivery to a patient, and the determined structure label mapS_(j) may be provided to treatment planning/delivery software 115. Thissegmentation result may be used to adjust the radiotherapy treatmentplan. For that purpose, the segmentation (method 500) may be performedimmediately prior to the upcoming radiotherapy treatment session, or oneday prior to the upcoming radiotherapy treatment session.

FIG. 6 is a flowchart illustrating an exemplary training method 600 fortraining a structure classifier model using an atlas. In someembodiments, method 600 may be used to implement step 506 of FIG. 5described above. In some embodiments, method 600 may be performed byclassifier training unit 302. The structure classifier model may be aRandom Forests model, such as the classifier model M_(j) of FIG. 5.

As shown in FIG. 6, at 601, classifier training unit 302 may receive anatlas that includes a training image and a corresponding structure labelmap indicating the structures that the image points of the trainingimage belong to. In some embodiments, the atlas may be a registeredatlas, such as atlas I_(j) ^((j-1)), S_(j) ^((j-1)).

At 602, feature extraction module 322 may select a plurality of trainingsamples from the mapped atlas image of each training atlas. Eachtraining sample can correspond to a single image point or a group ofimage points (such a group of image points is also referred to as asuper image point). According to the disclosure, the training samplesfrom a mapped atlas image can include all or a portion of the imagepoints on the mapped atlas image. When only a portion of the imagepoints are used for training, a sample selection can be performed todetermine what image points are used. For example, the training samplescan be selected fully randomly over the entire mapped atlas image, or beselected from a region within a certain distance to the border of thestructure of interest. As another example, the sample selection can beguided by the registration results such that more samples can beselected from an ambiguous region, i.e., the region where structurelabels from different mapped atlases do not completely agree with eachother or the disagreement is larger than a certain level (for example,three or more out of ten mapped atlases have a different determinationthan the other mapped atlases).

At 603, feature extraction module 322 may determine at least one imagefeature for an image point. Various types of features can be extracted,such as, for example, image intensity value, image location, imagegradient and gradient magnitude, eigen-values of a Hessian matrix of theimage, image texture measures such as energy, entropy, contrast,homogeneity, and correlation of local co-occurrence matrix, local imagepatches of varying sizes. Alternatively, attributes or features may alsobe automatically and adaptively computed using machine learning models.For example, a convolutional neural network model may be trained toextract relevant features from sample images, and the pre-trained modelcan be applied to the training samples to produce attributes. Aconvolutional neural network typically includes several convolutionlayers, among other layers, that produce feature maps of various sizes.The feature maps contain generic features characterizing the input image(or a selected portion of the input image), and thus can be used asfeatures in the structure classifier to further improve classificationresults. Features from various convolution layers (e.g., top layers,middle layers, lower layers), or a selection of these layers, may beused. In some embodiments, computation of attributes can be omitted ifthe training atlases already include the attributes for the atlas imagepoints that are to be used by the machine learning algorithm.

At 604, training module 323 may apply a learning algorithm to generatean anatomical classification model based on the identified imagefeatures for the image point. The machine learning algorithm can be asupervised learning algorithm, which seeks to infer a prediction modelgiven a set of training data. For example, the machine learningalgorithm for training the structure classifier can be the randomforests (RF) machine learning algorithm, which can naturally handlemultiple classes, i.e., one classifier to classify several structures.The output of an RF classifier can be a probability estimation of whichclass the input data belongs to, i.e., which structure the correspondingimage point belongs to. At 605, classifier training unit 302 may checkif all training samples have been process. If so (605: yes), method 600may proceed to 606, where classifier training unit 302 outputs thetrained classifier model, e.g., to be used by steps 507-508 of method500. Otherwise (605: no), method 600 may return to 601 to process thenext training sample.

Alternative implementations of methods 500 and 600 are contemplated.

In an embodiment, the registered prior day image may be replaced by thecurrent day image for training purpose. Accordingly, the training can bedescribed as:

$\left. M_{n}\leftarrow{{Train}\left\{ {\bigcup\limits_{j = 1}^{n - 1}\left( {I_{j},S_{j}^{({j - 1})}} \right)} \right\}} \right.,{n \geq 2},{n = {{current}\mspace{14mu} {day}}}$

In another embodiments, an exemplary image segmentation process usesprior day images and expert structures or contours C to segmentsucceeding-days images, without utilizing image registration. The expertdrawn or edited contours C are distinguished from the atlas segmentationcontours S (e.g., the ABAS structures have been edited by an expert).The exemplary training process of model M_(n) from the union of dataI_(j), C_(j) at treatment day n is described below:

$\left. M_{n}\leftarrow{{Train}\left\{ {\bigcup\limits_{j = 1}^{n - 1}\left( {I_{j},C_{j}} \right)} \right\}} \right.,{n \geq 2},{n = {{current}\mspace{14mu} {day}}}$

In another embodiment, an exemplary image segmentation process utilizesmethod 500 disclosed in FIG. 5, but replaces the prior days' ABASstructures S with that day's expert structures C. When the processreaches the last prior day of the patient's prior day images, the methoduses ABAS structures from the previous day. The exemplary trainingprocess of model RFn is described below:

$\left. M_{n}\leftarrow{{Train}\; \left\{ {{\bigcup\limits_{j = 1}^{n - 1}\left( {I_{j},C_{j}} \right)},\left( {I_{n},S_{n}^{({n - 1})}} \right)} \right\}} \right.,{n \geq 2},{n = {{current}\mspace{14mu} {{day}.}}}$

Learning algorithms other than RF methods may also be used incombination with ABAS segmentations in similar manners as describedabove. For example, the classifier model may a convolutional neuralnetwork model. A convolutional neural network may include a stack ofdistinct layers that transform an input image into an output structurelabel map. The layers may form two stages: an encoding stage and adecoding stage. The layers may differ in input size, output size, andthe relationship between the input and the output for the layer. Eachlayer may be connected to one or more upstream and downstream layers inthe stack of layers. The performance of a convolutional neural networkmay thus depend on the number of layers, and the convolutional neuralnetwork's complexity may increase as the number of layers increases. Aconvolutional neural network may be viewed as “deep” if it has more thanone stages of non-linear feature transformation, which typically meansthe number of layers in the network is above a certain number. Forexample, some convolutional neural networks may include about 10-30layers, or in some cases more than a few hundred layers. Examples ofconvolutional neural network models include AlexNet, VGGNet, GoogLeNet,ResNet, etc. These convolutional neural network models can be used atthe encoding stage of the full convolutional neural network model.

This present disclosure is not limited to the embodiments discussedabove, for other embodiments of training data combinations are possible.Models could be weighted to de-emphasize the population, for instance,or to delete it all together. The individual day's images and structurescould be trained in single-image RF models that could later be merged.Other combinations of registered and non-registered sets ofimages/structures may be possible. In addition, context features may beused in addition to the appearance features described above.

Various operations or functions are described herein, which may beimplemented or defined as software code or instructions. Such contentmay be directly executable (“object” or “executable” form), source code,or difference code (“delta” or “patch” code). Software implementationsof the embodiments described herein may be provided via an article ofmanufacture with the code or instructions stored thereon, or via amethod of operating a communication interface to send data via thecommunication interface. A machine or computer readable storage mediummay cause a machine to perform the functions or operations described,and includes any mechanism that stores information in a form accessibleby a machine (e.g., computing device, electronic system, and the like),such as recordable/non-recordable media (e.g., read only memory (ROM),random access memory (RAM), magnetic disk storage media, optical storagemedia, flash memory devices, and the like). A communication interfaceincludes any mechanism that interfaces to any of a hardwired, wireless,optical, and the like, medium to communicate to another device, such asa memory bus interface, a processor bus interface, an Internetconnection, a disk controller, and the like. The communication interfacecan be configured by providing configuration parameters and/or sendingsignals to prepare the communication interface to provide a data signaldescribing the software content. The communication interface can beaccessed via one or more commands or signals sent to the communicationinterface.

The present disclosure also relates to a system for performing theoperations herein. This system may be specially constructed for therequired purposes, or it may comprise a general-purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a computerreadable storage medium, such as, but not limited to, any type of diskincluding floppy disks, optical disks, CDROMs, and magnetic-opticaldisks, read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, or any type of media suitable forstoring electronic instructions, each coupled to a computer system bus.

The order of execution or performance of the operations in embodimentsof the present disclosure illustrated and described herein is notessential, unless otherwise specified. That is, the operations may beperformed in any order, unless otherwise specified, and embodiments ofthe present disclosure may include additional or fewer operations thanthose disclosed herein. For example, it is contemplated that executingor performing a particular operation before, contemporaneously with, orafter another operation is within the scope of aspects of the presentdisclosure.

Embodiments of the present disclosure may be implemented withcomputer-executable instructions. The computer-executable instructionsmay be organized into one or more computer-executable components ormodules. Aspects of the present disclosure may be implemented with anynumber and organization of such components or modules. For example,aspects of the present disclosure are not limited to the specificcomputer-executable instructions or the specific components or modulesillustrated in the figures and described herein. Other embodiments ofthe present disclosure may include different computer-executableinstructions or components having more or less functionality thanillustrated and described herein.

When introducing elements of aspects of the present disclosure or theembodiments thereof, the articles “a,” “an,” “the,” and “said” areintended to mean that there are one or more of the elements. The terms“comprising,” “including,” and “having” are intended to be inclusive andmean that there may be additional elements other than the listedelements.

Having described aspects of the present disclosure in detail, it will beapparent that modifications and variations are possible withoutdeparting from the scope of aspects of the present disclosure as definedin the appended claims. As various changes could be made in the aboveconstructions, products, and methods without departing from the scope ofaspects of the present disclosure, it is intended that all mattercontained in the above description and shown in the accompanyingdrawings shall be interpreted as illustrative and not in a limitingsense.

What is claimed is:
 1. A system for segmenting medical images, thesystem comprising: a database configured to store a plurality of medicalimages acquired by an image acquisition device, including at least onefirst medical image of an object, and a second medical image of theobject, each first medical image associated with a first structure labelmap; and a processor, configured to: register the at least one firstmedical image to the second medical image; determine a classifier modelusing the registered first medical image and the corresponding firststructure label map; and determine a second structure label mapassociated with the second medical image using the classifier model. 2.The system according to claim 1, wherein the at least one first medicalimage includes a set of prior day images of the object.
 3. The systemaccording to claim 1, wherein the second medical image is a current dayimage of the object.
 4. The system according to claim 1, wherein thefirst structure label map includes expert structure labels identifiedfor the first medical image.
 5. The system according to claim 1, whereinthe processor is further configured to determine the first structurelabel map for the first medical image using a population-trainedclassifier model.
 6. The system according to claim 1, wherein theprocessor is further configured to register the first structure labelmap to the second medical image using an atlas-based segmentationmethod, and determine a classifier model using the registered firstmedical image and the registered first structure label map.
 7. Thesystem according to claim 1, wherein the classifier model is a RandomForests model.
 8. The system according to claim 1, wherein theclassifier model is a convolutional neural network model.
 9. The systemaccording to claim 1, wherein the processor is further configured toidentify at least one feature in the second medical image, and apply theclassifier model to the at least one feature.
 10. The system accordingto claim 9, wherein the at least one feature is computed using apre-trained convolutional neural network.
 11. The system according toclaim 1, wherein the at least one first medical image and the secondmedical image are acquired during serial radiotherapy treatment sessionsof a patient.
 12. The system according to claim 1, wherein the processoris configured to determine the second structure label map prior to aradiotherapy treatment delivery.
 13. A computer-implemented method forsegmenting medical images, the method comprising the followingoperations performed by at least one processor: receiving at least onefirst medical image of an object, and a second medical image of theobject, from a database configured to store a plurality of medicalimages acquired by an image acquisition device, each first medical imageassociated with a first structure label map; registering the at leastone first medical image to the second medical image; determining aclassifier model using the registered first medical image and thecorresponding first structure label map; and determining a secondstructure label map associated with the second medical image using theclassifier model.
 14. The method according to claim 13, wherein the atleast one first medical image includes a set of prior day images of theobject.
 15. The method according to claim 13, wherein the second medicalimage is a current day image of the object.
 16. The method according toclaim 13, further comprising determining the first structure label mapfor the first medical image using a population-trained classifier model.17. The method according to claim 13, the first structure label map isregistered to the second medical image using an atlas-based segmentationmethod.
 18. The method according to claim 13, further comprisingidentifying at least one feature in the second medical image, and applythe classifier model to the at least one feature.
 19. The methodaccording to claim 13, wherein the second structure label map isdetermined prior to a radiotherapy treatment delivery.
 20. Anon-transitory computer-readable medium containing instructions that,when executable by a processor, cause the processor to perform a methodfor segmenting medical images, the method comprising: receiving at leastone first medical image of an object, and a second medical image of theobject, from a database configured to store a plurality of medicalimages acquired by an image acquisition device, each first medical imageassociated with a first structure label map; registering the at leastone first medical image to the second medical image; determining aclassifier model using the registered first medical image and thecorresponding first structure label map; and determining a secondstructure label map associate with the second medical image using theclassifier model.